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LARGE-SAMPLE  THEORY  FOR  STANDARDIZED  TIME  SERIES:  AN  OVERVIEW 


Peter  W.  Glynn  and  Donald  L.  Iglehart 
ABSTRACT 

There  are  two  basic  approaches  to  constructing  confidence  intervals 
for  steady-state  parameters  from  a  single  simulation  run.  The  first  is  to 
consistently  estimate  the  variance  constant  in  the  relevant  central  limit 
theorem.  This  is  the  approach  used  in  the  regenerative,  spectral,  and 
autoregressive  methods.  The  second  approach  (standardized  time  series, 

STS)  due  to  SCHRUBEN  [10]  is  to  “cancel  out"  the  variance  constant.  This 
second  approach  contains  the  batch  means  method  as  a  special  case.  Our 
goal  in  this  paper  is  to  discuss  the  large-sample  properties  of  the 
confidence  Intervals  generated  by  the  STS  method.  In  particular,  the 
asymptotic  (as  run  size  becomes  large)  expected  value  and  variance  of  the 
length  of  these  confidence  Intervals  is  studied  and  shown  to  be  inferior  to 
the  behavior  manifested  by  intervals  constructed  using  the  first  approach. 

-  Key  Words:  batch  means 

confidence  intervals 

functional  central  limit  theorem 

weak  convergence  of  probability  measures 

simulation  output  analysis 

standardized  time  series 


steady-state  simulation. 

V 


1.  INTRODUCTION 


Standardized  time  series  has  attracted  considerable  attention  in  the 
simulation  community,  since  its  introduction  by  SCHRUBEN  [10].  Although 
the  method  has  been  exploited  for  the  purpose  of  detecting  initialization 
bias  (see  [9]),  our  Intent  here  is  to  focus  solely  on  the  application  of 
standardized  time  series  to  generation  of  confidence  intervals  for 
steady-state  simulations.  We  shall  describe,  for  these  confidence 
intervals,  certain  large-sample  results  first  obtained  in  GLYNN  and 
IGLEHART  [5].  We  start,  in  Section  2,  by  outlining  the  steady-state 
confidence  interval  problem.  In  Section  3,  the  method  of  standardized  time 
series  is  described,  while  Section  4  is  devoted  to  the  large-sample  theory 
for  the  confidence  Intervals  generated.  Section  5  concludes  with  a 
description  of  future  research  problems  associated  with  the  method  of 
standardized  time  series. 


2.  THE  STEADY-STATE  CONFIDENCE  INTERVAL  PROBLEM 

Let  Y  »  (Y(t)  :  t  _>  0}  be  a  real-valued  stochastic  process 
representing  the  output  of  a  simulation.  (To  Incorporate  stochastic 
sequences  (Yq  :  n  ^  0}  into  our  framework,  set  Y(t)  *  Yj t ] »  where  [tj 
is  the  greatest  integer  less  than  or  equal  to  t.)  Assume  that  Y 
possesses  a  steady-state,  in  the  sense  that 


r(t)  =  —  /  Y(s)ds  r 
c  0 


(2.1) 


as  t  ♦  ®  ("►  denotes  weak  convergence) ,  where  r  is  a  finite 

deterministic  constant. 
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The  steady— state  estimation  problem  of  simulation  has  basically  two 
parts: 

i)  construct  a  consistent  estimator  for  the  steady-state  parameter  r 
11)  assess  the  variability  of  the  estimator  constructed  in  1). 

Given  (2.1),  the  solution  to  part  1)  is  trivial:  namely,  given  an 
observation  of  the  process  T  over  the  interval  {0,t) ,  use  r(t)  (or 
some  asymptotically  equivalent  variant  thereof)  as  our  point  estimator 
for  r.  This  method  for  estimating  r  is  known  as  a  single- 
replicate  method.  Although  multiple  replicate  procedures  for  estimating  r 
are  also  available  (see  GLYNN  [4]  for  the  relevant  large-sample  theory),  we 
shall  restrict  our  attention  here  to  single-replicate  methods. 

Part  ii)  is  the  crux  of  the  steady-state  estimation  problem;  it  has 
attracted  a  great  deal  of  attention  in  the  simulation  literature  over  the 
last  ten  years.  The  traditional  variability  assessment  technique,  which 
has  been  widely  adopted  in  the  simulation  community,  is  to  construct 
confidence  intervals  for  the  point  estimator.  It  should  be  emphasized, 
however,  that  there  are  other  possible  methods  available  for  assessing 
point  estimator  variability.  For  example,  one  could  use  an  estimate  of  the 
mean  square  error  (MSE)  of  the  point  estimator  for  the  same  purpose. 

In  any  case,  confidence  interval  generation  has  generally  been  regarded  as 
the  most  reasonable  means  for  assessing  point  estimator  variability. 
Adopting  this  viewpoint,  most  published  analyses  have  required  assuming  a 
strengthened  version  of  (2.1),  namely  existence  of  a  finite  positive 


constant  o  such  that 


as  t  -*■  ”,  where  N(0,1)  is  a  normal  random  variable  (RV)  with  mean  zero 
and  unit  variance. 

To  use  the  central  limit  theorem  (CLT)  (2.2)  directly  to  generate 
confidence  Intervals,  one  needs  a  process  (s(t)  :  t  ^  0},  with  the 
following  properties: 

1)  s(t)  may  be  constructed  from  the  process  T  observed  over 
the  interval  [0,t] 

ii)  s(t)  a  as  t  •*  ®.  (2.3) 

Property  (2.3)  ii)  says  that  s(t)  consistently  estimates  the  parameter 
a.  Given  such  a  process  (s(t)  :  t  _>  0},  the  interval 

[r(t)  -  z&  r(t)  +  z& 

can  easily  be  shown  to  be  an  asymptotic  100( 1-6)Z  confidence  Interval  for 

r,  provided  that  z^  is  chosen  as  a  solution  to  P{N(0,1)  <  z^>  -  1-6/2. 

Thus,  under  the  hypothesis  (2.2),  the  confidence  Interval  generation 

problem  is  reduced  to  construction  of  consistent  estimators  for  a.  We 

will  call  confidence  Interval  procedures  which  follow  this  line  of 

attack  consistent  estimation  procedures. 

Note  that  a  has  an  Important  statistical  interpretation.  If  the 
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process  (t(r(t)-r)  :  t  ^  0}  is  uniformly  integrable  (this  should  be 

viewed  as  a  mathematical  regularity  condition  which,  from  a  practical 
viewpoint,  is  just  slightly  stronger  than  the  assertion  (2.2)),  then  we  can 
pass  expectations  through  the  limit  theorem  obtained  by  squaring  both  sides 
of  (2.2),  thereby  yielding 


t  E(r(t)-r)  -»>  a 


as  t  ■*  ®.  In  other  words,  the  MSE  of  the  point  estimator  r(t)  is 
2 

asymptotic  to  a  /t.  Thus,  a  consistent  estimation  procedure  not  only 

produces  a  confidence  interval  for  r,  but  also  an  estimate  for  the  MSE  of 
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the  point  estimator  r(t),  namely  s  (t)/t. 

We  view  the  following  well-known  techniques  as  consistent  estimation 
procedures: 

i)  the  regenerative  method 

il)  spectral  procedures  (with  bandwidth  going  to  zero  as  sample  size 
t  >  ®) 

iii)  autoregressive  methods  (with  model  order  going  to  infinity  as  sample 
size  t  '  <*>) 

v)  batch  means  (with  both  the  batch  size  and  number  of  batches  going  to 
infinity  as  sample  size  t  -*■  ®). 

Mathematical  theory  verifying  consistency  of  the  regenerative  estimator  is 
available  (see,  for  example,  IGLEHART  (8]);  we  are  not,  however,  aware  of 
such  consistency  results  for  the  other  three  techniques  mentioned  above,  in 
the  context  of  the  non-s tat ionary  processes  which  arise  in  simulation. 

A  totally  different  approach  to  the  confidence  interval  generation 
problem  is,  however,  available.  A  glance  at  the  limit  theorem  (2.2) 
indicates  that  a  acts,  in  some  sense,  as  a  scaling  parameter.  This 
suggests  that  one  can  eliminate  a  entirely  from  the  limit  theorem  (2.2) 
by  passing  to  a  "dimensionless"  version  of  the  result:  this  is  the  basic 
idea  behind  standardized  time  series. 


More  specifically,  suppose  Chat  one  can  find  a  non-vanishing  process 


iZ(t)  :  t  ^  0}  such  that 

(t1/2(r(t)-r),  Z(t))  (aN(0,l),  aZ)  (2.4) 

as  t  ♦  it  follows,  from  the  continuous  mapping  theorem  (see  p.  31  of 
BILLINGSLEY  [lj),  that 

t1/2(r(t)-r)/Z(t)  N(0,1)/Z  (2.5) 


as  t  >  ®. 

At  first  glance,  it  appears  that  (2.5)  should  be  enough  to  construct  a 
confidence  Interval  for  r.  However,  it  is  clear  that  one  possible  choice 
for  (Z(t)  :  t  ^  0}  is  Z(t)  =  o.  While  this  is,  of  course,  a 
mathematically  reasonable  possibility,  it  cannot  be  Implemented  from  a 
practical  viewpoint  since  Z(t)  depends  on  the  unknown  parameter  o. 

One  might  also  attempt  to  use  processes  (Z(t)  :  t  _>  0},  in  which 
Z(t)  -  Z(t,r)  is  a  function  of  the  parameter  r.  Clearly,  then,  Z(t,r) 
must  have  a  very  special  dependence  on  r,  in  order  that  the  limit  theorem 
(2.4)  may  be  "unfolded"  to  reveal  a  confidence  interval  for  r  of  the 
form: 


[r(t)  - 


L(t) 

1 1/ 2  * 


r(t) 


U(t)  i 

172-1  » 


(2.6) 


when  L(t),  U(t)  do  not  depend  on  r  (or  any  other  unknown  parameters). 

To  the  best  of  our  knowledge,  the  only  confidence  interval  of  the  form 
(2.6),  in  which  the  scaling  process  Z(t)  depends  explicitly  on  r,  is  the 


5 


Fieller  Interval  (see  IGLEHART  [7J);  the  construction  of  this  interval  is 
possible  only  because  of  the  quadratic  dependence  of  Z(t)  on  r. 

In  light  of  the  above  discussion,  it  seems  reasonable  to  limit 
ourselves  to  processes  (Z(t)  :  t  0}  such  that: 

Z(t)  does  not  depend  on  any  unknown  parameters.  (2.7) 

Because  (2.7)  is  not  mathematically  precise,  we  shall  need  to  describe  this 
assumption  in  more  detail  in  the  next  section. 

In  any  case,  (2.4)  and  (2.7)  form  the  core  of  the  method  of 
standardized  time  series.  In  view  of  (2.5),  we  shall  call  procedures  based 
on  (2.4)  and  (2.7)  cancellation  methods.  If  we  adopt  the  convention  that 
Z  cannot  be  deterministic,  then  cancellation  methods  and  consistent 
estimation  procedures  are  disjoint  families  of  techniques,  and  we  arrive  at 
a  decomposition  of  steady-state  confidence  interval  methods  which  is 
described  by  Figure  1. 

3.  STANDARDIZED  TIME  SERIES 

As  indicated  in  the  previous  section,  standardized  time  series  is  a 
cancellation  method;  this  point  was  made  by  SCHRUBEN  in  [10].  Recall  that 
a  is  a  measure  of  the  variability  of  r(t)-r.  Thus,  if  Z(t)  is  to  be 
proportional  to  a ,  it  seems  reasonable  for  Z(t)  to  look  at  the 
fluctuations  of  r(t)-r  as  a  function  of  time.  This  suggests  that  one 
should  base  one's  analysis  on  the  process  r(*)-r,  as  opposed  to  merely  the 
point  estimate  r(t)-r. 


Figure  1 

In  keeping  with  this  viewpoint,  it  is  natural  to  extend  the  CLT  (2.2) 
to  a  functional  central  limit  theorem  (FCLT).  A  FCLT  provides  a 
distributional  approximation  for  the  entire  process  r(»)-r.  To  be  more 
precise,  let 


i  nc 

Y  (t)  -  -M  Y(s)ds , 


0  <  t  <  1  . 


Given  that  T  possesses  a  steady-state  (see  (2.1)),  it  follows,  upon 
observing  that  Yn(t)  "  t  r(nt),  that 
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Y  (t)  =■»  rt 
n 

as  n  -*•  ®.  To  obtain  some  sense  of  the  random  fluctuations  of  Y  (t) 

n 

around  rt,  it  is  necessary  to  "scale  up”  the  difference  Yn(t)  ~  ft*  To 

be  consistent  with  the  CLT  (2.2),  the  scaling  factor  must  clearly  be 
1/2 

n  .So,  consider  the  random  function 

X(t)  =  n1/2(Y  (t)  -  rt)  , 
n  n 

for  0  <  t  <  1.  A  FCLT  provides  an  approximation,  in  distribution,  for  the 

process  X  *{X(t):0<t<l}. 

□  n  —  — 

To  determine  the  limit  X  of  the  sequence  Xq,  observe  that  the 

continuity  of  Xr  suggests  that  the  limit  X  ought  to  be  continuous  in 

t.  Furthermore,  consider  the  increments  of  X  : 

n 

.  n(t+s) 

X  (t+s)  -  X  (t)  -  n1/z(i  /  Y(u)du  -  rs) 

n  n  n  ' 

nt 

Xn(t)  "  Xn(0)  "  "1/2^n  /  Y<u>du  “  rt)  •  (3.1) 

Note  that  X  (t)  -  X  (0)  a  X  (t-c)  -  X  (0)  for  e  small  and  positive, 
n  n  n  n 

From  (3.1),  it  follows  that  X^(t+s)  -  Xn(t)  depends  only  on  the  evolution 

of  T  after  time  nt,  whereas  Xn(t-e)  -  Xn(0)  depends  only  on  T  up  to 

time  nt-ne.  As  n  +  ®,  one  expects  that  events  occurring  ne  time  units 

apart  are  asymptotically  independent;  consequently,  the  limit  process  X 

should  have  independent  increments,  on  the  sense  that  for  t^  <  t2  <  •••  < 

t  ,  the  random  variables  X(t0)  -  X(t,),  ...,  X(t  )  -  X(t  ,)  are 
n  t  i  n  n- 1 
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independent .  (Extending  our  heuristic  argument  above  from  two  increments 
to  n  increments  is  easy.)  Finally,  if  the  process  T  is  time- 
homogeneous,  it  makes  sense  (see  (3.1)),  that 

X  (t+s)  -  X  (t)  i  X  (s)  -  X  (0) 
n  n  n  n 

g) 

(  ~  means  "has  approximately  the  same  distribution  as").  Passing  to  the 
limit  we  therefore  expect  that  X  will  have  stationary  increments,  in  the 
sense  that  X(t+s)  -  X(t)  X(s)  -  X(0).  (fl>  denotes  equality  in  distribu¬ 
tion.  ) 

To  summarize,  we  expect  that  the  limit  process  X: 
i)  is  continuous , 

li)  has  independent  increments,  and 
iii)  has  stationary  increments.  (3.2) 

It  turns  out  that  (3.2)  uniquely  characterizes  the  possible  limit  X; 
namely,  X  must  be  a  Brownian  motion  (see  BREIMAN  [2],  Chapter  12,  for 
definitions  and  results).  A  Brownian  motion  process  must  necessarily  be 
Gaussian,  which  is  consistent  with  the  CLT  (2.2). 

We  can  now  state  our  FCLT  assumption  on  T: 

X  (•)  <jB( •)  (3.3) 

n 

as  n  ♦  ®,  where  B(«)  is  a  standard  Brownian  motion  (i.e.,  B(0)  “  0, 
var  B(t)  ■  t).  Note  that  (3.3)  refers  to  weak  convergence  of  the  random 
function  xn(*)  to  the  random  function  aB(»).  Thus,  our  FCLT  assumption 


requires  an  understanding  of  the  notion  of  weak  convergence  of  random 
functions,  as  opposed  to  the  more  standard  notion  of  weak  convergence  of 
random  variables.  We  shall  not  go  Into  further  detail  on  this  point, 
except  to  mention  that  a  brief  discussion  of  the  relevant  convergence 
issues  is  given  in  Section  2  of  GLYNN  and  IGLEHART  [5]. 

Of  more  practical  Importance  is  the  need  to  have  an  understanding  of 
when  a  FCLT  holds.  As  our  heuristic  argument  indicates,  one  basically 
needs  a  time-homogeneous  process  with  an  appropriate  asymptotic 
independence  structure.  To  be  more  concrete,  assume  that  T  is  derived 
from  a  stochastic  sequence  obeying  a  recursion  of  the  form 


Vi  * h(o-  v  vi> 


(3.4) 


where  t)q+1  is  Independent  of  Yq,  Y^ ,  ...,  Yn»  In  order  that  T  be 
time-homogeneous,  it  is  necessary  that  the  t^'s  be  identically 
distributed  and  that  h(n,y,q)  not  depend  explicitly  on  n. 

From  a  simulation  standpoint,  the  time-homogeneity  requirement  can  be 
stated  as: 


The  state  transition  rules  and  clock-resetting  rules  should  not 
contain  any  explicit  time-dependence. 

Thus,  a  simulation  which  explicitly  incorporates  time  trends  is  not 
time-homogeneous . 

The  asymptotic  independence  assumption  is  easily  understood,  from  an 
intuitive  standpoint:  events  in  the  distant  past  should  not  affect  events 
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in  Che  distant  future  too  much.  Mathematically,  this  requirement  is  often 
stated  in  terms  of  mixing  conditions. 

♦-mixing:  This  is  a  very  restrictive  type  of  mixing  condition,  which 
basically  says  that 

j P(A| B)  -  P(A) | 

should  be  uniformly  small  in  both  A  and  B,  provided  that  A  and  B  are 
events  separated  by  a  long  time  period.  (See  p.  166-167  of  BILLINGSLEY  [ 1 J 
for  a  rigorous  definition.) 

Strong  mixing:  This  is  a  less  restrictive  type  of  mixing  condition, 
stating  that 

{ PC A  n  B)  -  P(A)P(B) | 

ought  to  be  uniformly  small  in  both  A  and  B,  provided  that  A  and  B 
are  events  separated  by  a  long  time  period.  (See  HALL  and  HEYDE  [6]  for  a 
definition. ) 

Typically,  in  order  to  obtain  a  FCLT,  certain  rate  conditions  are 
imposed  on  the  mixing  coefficients;  these  conditions  tend  to  be  more 
stringent  for  strong  mixing  processes  than  ♦-mixing  processes. 

It  turns  out  that  regenerative  processes  that  are  positive  recurrent 
(i.e.,  finite  expected  time  between  regenerations)  are  automatically  strong 
mixing.  (See  GLYNN  [3]  for  a  proof.)  For  a  regenerative  process  to  be 
^-mixing,  it  is  necessary  that  regenerations  occur  "uniformly  fast” 


throughout  the  state  space.  More  precisely,  if  the  regenerative 
process  is  time-homogeneous  Markov  on  a  state  space  S,  then  we  require 
that  there  exist  n  1  such  that 

sup{P{T  >  njYQ  -  x}  :  x  £  S}  <  1  ,  (3.5) 

where  T  is  the  first  regeneration  time  for  Y.  Note  that  (3.5) 
disqualifies  any  birth-death  type  process  from  being  ^-mixing,  since  the 
hitting  time  of  any  fixed  state  cannot  be  uniformly  bounded;  in  fact,  the 
hitting  time  of  any  fixed  state  tends  to  infinity  as  the  initial  state  goes 
to  infinity.  By  similar  reasoning,  virtually  any  simulation  of  an  open 
queueing  network  (with  an  unlimited  number  of  potential  customers)  will 
give  rise  to  a  process  which  is  not  ^-mixing.  (However,  such  processes 
will  frequently  be  strong  mixing.) 

In  terms  of  mathematical  conditions  guaranteeing  that  Y  satisfy  the 
FCLT  (3.3),  the  reader  is  referred  to  Section  3  of  [5].  It  turns  out  that, 
mathematically  speaking,  one  "almost  always”  gets  a  FCLT  whenever  a  CLT 
holds,  so  that  the  FCLT  assumption  is  only  slightly  stronger  than  the  CLT 
assumption.  From  a  practical  standpoint,  we  repeat  that  an  FCLT  holds 
whenever  one  has  time-homogeneity  and  asymptotic  Independence,  asymptotic 
independence  usually  being  described  by  mixing  hypotheses. 

We  now  need  to  discuss  how  the  FCLT  (3.3)  is  used  to  obtain  the 
scaling  process  (Z(t)  :  t  ^  0}.  Let  g  be  a  real-valued  function  defined 
on  a  function  space,  so  that  g(X^)  makes  sense  as  a  random  variable. 
Assume  that: 

i)  g(ax)  -  ag(x)  for  all  a  >  0  and  functions  x  (this 
is  the  scaling  property  for  Z(t)). 


li)  g(B)  >  0  w.p.l  (this  guarantees  that  Z(t)  is 
non-vanishing  for  t  sufficiently  large) 


(3.6) 


iii)  g(x  +  pk)  -  g(x)  for  all  p,  where  k(t)  ■  t  (x  +  pk 
is  the  function  whose  value  at  t  is  x(t)  +  pk(t)). 

This  guarantees  that  g(Xn)  does  not  depend 
implicitly  on  the  unknown  parameter  r  (see  (2.7)). 

In  addition,  we  want  g  to  be  suitably  continuous,  but  we  shall  not  be 
precise  here  about  this.  (See  Section  3  of  [5]  for  details.) 

Let  h(x)  ■  x(l)/g(x).  By  the  continuous  mapping  lemma  (this  is  where 
we  need  g  to  be  suitably  continuous),  and  (3.3) 


h(X  )  ^  h(oB)  , 
n 


as  n  ®.  But  h(oB)  -  oB(l)/g(aB)  ■  oB(l)/ag(B)  ■  B(l)/g(B),  so  that  the 
a  has  ‘'cancelled  out.”  On  the  other  hand. 


h(xn) 


n1/2(Yn(l)  -  p) 

g(n1/r2(Y  -  kp)) 

n 


(Yn<l)  -  „) 

g(Y„  -  kn) 


V‘>  - 1*  _ 

g(Yn) 


So,  assuming  that  Y  satisfies  an  FCLT  and  g  satisfies  (3.6),  the 


Interval 


(3.7) 


[Yn(l)  -  z( l-a/2)  g(Yn)  ,  Yn(l)  -  z(a/2)  g(YR)] 

is  an  asymptotic  100  (1-6)%  confidence  interval  for  r,  where  H(z(a))  *  a, 

and  H( • )  ■  P{B(l)/g(B)  <  •)  (We  implicitly  assumed  here  that  H  was  a 

strictly  increasing  continuous  distribution  function,  but  this  turns  out  to 

be  justified;  see  Section  3  of  [5]). 

The  process  (Y  -  rk)/g(Y  )  is  called  a  standardized  time  aeries, 
n  n 

Letting  31  be  the  set  of  g's  satisfying  (3.6),  we  find  that  each  g  e 31 
gives  rise  to  a  standardized  time  series.  Among  possible  choices  for  g 
are: 

i)  batch  means  with  the  number  of  batches  fixed  as  sample  size 
tends  to  infinity  (in  other  words,  there  exists  a  choice  of  g 
giving  rise  to  batch  means  confidence  Intervals), 
ii)  Schruben’s  standardized  sum  process  (see  [10]). 
ill)  Schruben's  standardized  maximum  process  (see  [10]). 

For  more  detail  on  possible  choices  for  g,  see  Section  4  of  [5]. 

It  should  be  noted  that  in  order  to  Incorporate  the  method  of  batch 
means,  with  the  number  of  batches  increasing  with  the  sample  size,  it  is 
necessary  to  consider  Z(t)'s  of  the  form  Sn(xn)>  where  gn  e3i.  One  then 
needs  to  examine  weak  convergence  of  Xn(l)/gn(Xn);  this  type  of  weak 
convergence  problem  tends  to  be  much  more  complicated  to  analyze  math¬ 
ematically  than  the  case  in  which  g,J(*)  Is  independent  of  n. 

Fortunately,  with  our  definition  of  standardized  time  series,  there  is  no 
need  to  consider  such  convergence  problems. 


-  ■  »  •  • '  '•  -  V  -  , 


■.  V  *. 


.*•  *’•  .**»  *  ^  »*•  *'*  *•  ,*•  ,*• , 
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To  summarize  the  above  discussion,  we  have  shown  that  each  g  e 


gives  rise  to  a  standardized  time  series  confidence  interval  which  is 
asymptotically  valid. 

4.  FURTHER  PROPERTIES  OF  STANDARDIZED  TIME  SERIES 

As  indicated  in  Section  3,  each  standardized  time  series  procedure 
gives  rise  to  asymptotically  valid  confidence  intervals.  We  now  wish  to 
discuss  the  desirability  of  these  intervals.  The  discussion  of 
desirability  will  focus  on  two  issues: 

i)  asymptotic  mean  of  the  confidence  interval  half-length, 
ii)  asymptotic  variability  of  the  confidence  interval  half-length. 
Let  Ln(g)  be  the  length  of  the  confidence  interval  based  on  observing  T 
over  the  interval  [0,n]  and  using  a  standardized  time  series  based  on 
g  c  JW.  Clearly, 


I*n<g)  -  (z(l-o/2)  -  z(a/2))  g(*n)  , 


by  (3.7).  Noting  that 

g(Yn)  -  g(Yn  -  kr)  -  n~1/2  g(n1/2(Yn  -  kr)) 


-  n'1/2 


g(xn) 


f 


we  see  that 

n^2  Ln(g)  (z(l-a/2)  -  z(a/2))  ag(B)  , 

-1/2 

as  n  ®.  Thus,  L  (g)  is  of  order  n  .  As  is  clear  from  Section  2, 

n 

this  is  the  same  order  convergence  rate  as  that  obtained  for  consistent 
estimation  procedures. 


However,  a  more  precise  comparison  can  be  made.  Under  appropriate 
uniform  lntegrabllity  hypotheses  (this  should  be  viewed  as  a  mild 
additional  mathematical  assumption  on  T) ,  one  can  pass  expectations 
through  (4.1): 

n1/2  ELn(g)  -►  (z(l-a/2)  -  z(a/2)  •  aEg(B)  (4.2) 

as  n  ->  <».  Clearly,  one  would  like  to  select  g  aSM  so  that  the 
right-hand  side  of  (4.2)  Is  as  small  as  possible.  It  turns  out  that 

inf{z(l-a/2)  -  z(a/2)  Eg(B)  :  g  e  JK}  *  2zg  (4.3) 

where  Zg  Is  the  (1-5/2)%  quantile  of  a  N(0,1)  r.v.  (see  Section  5  of 
[5]).  The  right-hand  side  of  (4.3)  Is  the  asymptotic  confidence  Interval 
length  obtained  via  a  consistent  estimation  technique.  We  therefore  have 
that: 

Asymptotically,  a  standardized  time  series  confidence  Interval  is 
at  least  as  long,  in  expected  value,  as  a  confidence  Interval 
obtained  via  consistent  estimation. 

In  fact.  It  turns  out  that  standardized  time  series  confidence 
intervals  are  strictly  longer  than  consistent  estimation  Intervals,  because 
the  infinum  In  (4.3)  Is  not  attained  within  • 

What  about  asymptotic  variability  of  LQ(g)?  Note  that 


-  2 

n  var  g(Y^)  *  °  var  g(B) 

as  n  -*■  ®  (again,  provided  that  appropriate  uniform  integrability  for  Y 
is  available).  It  turns  out  that  if  g  e  3i,  then  g(B)  can  not  be 
deterministic  (this  justifies  calling  standardized  time  series  a 
cancellation  procedure;  see  Section  2),  so  var  g(B)  >  0.  Thus,  the 
variance  of  Ln(g)  *8  order  n  *. 

On  the  other  hand,  if  consistent  estimation  is  used,  we  claim  that  the 
variance  of  Ln(g)  is  of  smaller  order  than  n  *.  The  length  of  such  a 
confidence  interval  is  given  by 


so 


L  *  2  z.  s(n)/  ✓n  ; 
n  o 


var  L  -  —  4  zi  var  s(n) 
n  n  0 


But  s(n)  ^  a  as  n  ♦  ® ,  so  that  (under  appropriate  uniform 

integrability),  var  s(n)  •*>  var  a  -  0,  showing  that  var  is  of  smaller 

order  than  n  .  In  fact,  in  1 5 J  it  is  shown  that  for  the  regenerative 

-2 

method,  var  L  is  of  order  n  .  We  conclude  that: 
n 

Asymptotically,  a  standardized  time  series  confidence  interval 
is  more  variable  than  a  confidence  interval  obtained  via 
consistent  estimation. 


It  is  of  interest  to  return  to  a  philosophical  point  discussed  in  Section 
2.  It  was  mentioned  there  that  confidence  Intervals  are  but  one  way  to 
measure  the  variability  of  a  point  estimate.  Another  alternative  is  to 


just  estimate  the  MSE  of  the  point  estimate,  namely  7?  Clearly, 
consistent  estimation  techniques  give  such  an  estimate,  in  addition  to  a 
confidence  interval  for  r. 

However,  it  does  not  appear  that  standardized  ti  e  series  gives 
consistent  estimators  for  rr?  For  observe  that 

n1^2  g(Yn)  ?g(B)  . 

1/2  - 

Since  g(B)  is  non-deterministic  if  g  e  SH,  it  follows  that  n  gCY^) 
does  not  consistently  estimate  a. 

5.  SUMMARY  AND  FUTURE  WORK 

Schruben's  method  of  standardized  time  series  for  generating 
steady-state  confidence  intervals  has  provided  a  totally  new  approach  for 
attacking  one  of  the  hardest  problems  in  output  analysis.  It  appears  to 
behave  well  empirically,  and  seems  to  be  a  robust  procedure.  However,  it 
suffers  two  defects  from  a  mathematical  viewpoint:  longer  confidence 
intervals,  on  average,  and  more  variable  confidence  intervals,  than 
consistent  estimation  methods.  On  the  other  hand,  these  results  pertain 
only  to  large-sample  asymptotics,  and  therefore  it  is  possible  that 
standardized  time  series  procedures  may  strictly  dominate  consistent 
estimation  methods  in  a  small-sample  context. 

As  for  future  work  to  be  done  in  the  area,  the  following  problems  seem 
interesting: 

1.  Mathematical  analysis  of  small-sample  theory  for  standardized  time 
series  (this  would  involve  obtaining  Berry-Esseen  type  results  for 
FCLT's  and  would  be  hard). 


